{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fca72f1e",
   "metadata": {},
   "source": [
    "# How to Build a Video Segment Copy Detection System\n",
    "\n",
    "In the previous example we have demonstrated how to use milvus and towhee to build a simple video copy detection system at video level. In this tutorial, we will demonstrate video copy detection down to segment level. \n",
    "\n",
    "**What's the differents between video-level deduplication and segment-level deduplication?**\n",
    "\n",
    "\n",
    "- Video-level deduplication is a method for situations with high repetition. It finds duplicate videos by comparing the similarity between the embeddings of the whole video. Since only one embedding is extracted from a video, this method works faster. But the limitation of this method is also obvious: it is not good for detecting similar videos of different lengths. For example, the first quarter of video A and video B are exactly the same, but their embeddings may not be similar. In this case, it is obviously impossible to detect infringing content.\n",
    " \n",
    "- Segment-level deduplication detects the specific start and end times of repeated segments, which can handle complex clipping and insertion of video segments as well as situations where the video lengths are not equal. It does so by comparing the similarity between video frames. Obviously, we need to use this method in the actual task of mass video duplication checking. Of course, the speed of this method will be slower than the one of video level.\n",
    "\n",
    "**What are Milvus & Towhee?**\n",
    "\n",
    "- [Milvus](https://milvus.io/) is the most advanced open-source vector database built for AI applications and supports nearest neighbor embedding search across tens of millions of entries.\n",
    "- [Towhee](https://towhee.io/) is a framework that provides ETL for unstructured data using SoTA machine learning models.\n",
    "\n",
    "In this tutorial, we will demonstrate video duplication detection at segment level using Towhee and Milvus. Moreover, we managed to make the core functionality as simple as few lines of code, with which you can start hacking your own video deduplication engine.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a449ae79",
   "metadata": {},
   "source": [
    "## Preparation\n",
    "\n",
    "### Install packages\n",
    "\n",
    "Make sure you have installed required python packages:\n",
    "\n",
    "| package |\n",
    "| -- |\n",
    "| towhee |\n",
    "| pillow |\n",
    "| ipython |\n",
    "| numpy |\n",
    "| plyvel / happybase |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "23cb0567",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python -m pip install -q towhee pillow ipython numpy plyvel happybase"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8193915",
   "metadata": {},
   "source": [
    "### Prepare the data\n",
    "\n",
    "First, we need to prepare the dataset and Milvus environment.   \n",
    "\n",
    "The VCDB core dataset is almost all full video length repetition, which is not suitable for the evaluation of segment repetition detection technology. In this tutorial, we use [VCSL dataset](https://arxiv.org/abs/2203.02654). \n",
    "\n",
    "VCSL is a large-scale real dataset for video duplication detection from Youtube and Bilibili. Unlike VCDB, its cheating changes to video frames are more complex, including crop, filter, text overlay, background, cam-cording, picture in picture, even recent deepfake, etc. there are a wide range of content transformations among over 280k segment copies in VCSL, and these realistic skillful transformations bring great challenges to segment-level copy detection. \n",
    "\n",
    "In this tutorial, we use only a mini set of VCSL. It contains 5 events with three videos in each event, which are copies of each other. There is also a broken video in folder `crashed_video` for robustness testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3557e9a3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\n",
      "100  399M  100  399M    0     0  7072k      0  0:00:57  0:00:57 --:--:-- 10.7M    0  4762k      0  0:01:25  0:00:10  0:01:15 7573k  0     0  6269k      0  0:01:05  0:00:30  0:00:35 6113k6330k      0  0:01:04  0:00:32  0:00:32 7155k7  0:00:14 5008k\n"
     ]
    }
   ],
   "source": [
    "! curl -L https://github.com/towhee-io/examples/releases/download/data/VCSL-demo-en.zip -O\n",
    "! unzip -q -o VCSL-demo-en.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16dd5fd7",
   "metadata": {},
   "source": [
    "The directory structure of this demo dataset is like this:\n",
    "```\n",
    "./VCSL-demo-en/\n",
    "├── animation\n",
    "│   ├── 1b4e048011714eab928650f64e6370a6-123-JqSBkvPsYBw.mkv\n",
    "│   ├── 043bee7a71f347f18e8576bb1a01c86b-Langrisser-y3qAPvWnL18.mkv\n",
    "│   └── 1134884a6c544a00b3bce99457a2a98d-LANGRISSER_Mobile_OP-EA4PmSWGr8o.mp4\n",
    "├── baisui_mountain\n",
    "│   ├── 41c4eaced0d24ebba50d180026531025-Baisui_Mountain_commercial-1UJ411s7we.flv\n",
    "│   ├── 217a12c936414660a53b55b22e2aea59-20200607Baisui_Mountain-1kf4y127GB.flv\n",
    "│   └── 03584e404c0847fcbe5f9c486e8f8fc7-Baisui_Mountain_means_this-1UE411V74D\n",
    "├── madongmei\n",
    "│   ├── 8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv\n",
    "│   ├── 0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv\n",
    "│   └── ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv\n",
    "├── the_wandering_earth\n",
    "│   ├── d62ce5becff14a0c9c7dab5eea6647dc_the_wandering_earth_Wu_Jing_became_teenage_idol-1qf4y1G7gM.flv\n",
    "│   ├── e5dc80abd7a24b47accde190c9fdbcdc-The_Wandering_Earth_is_on_CCTV-1db411m7f8.flv\n",
    "│   └── ef65e0f662e646a88a13b6eddb640e48-News_Broadcast_on_Wandering_Earth-1xb411U7uE.flv\n",
    "└── yellow_alien\n",
    "    ├── 0fba4debd48245699971c4c18608cde3-Yellow_skinned_alien_low_cost_production-1vP4y1s79h.flv\n",
    "    ├── 9ec6f467472a428e91fd45bb688e3be1-drunken_square_yellow_alien-1uU4y1E7n6.flv\n",
    "    └── 01414786cbe74d2d95e2b71df26f39a3-Golden_Wheel_It_doesnt_matter_anymore-1sh411B7g1.flv\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b335d502",
   "metadata": {},
   "source": [
    "Define some helper function to convert video to gif so that we can have a look at these videos.   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "85deefa1",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython import display\n",
    "from pathlib import Path\n",
    "import towhee\n",
    "from towhee import pipe, ops\n",
    "from PIL import Image as PILImage\n",
    "import os\n",
    "\n",
    "def display_gif(video_path_list, text_list):\n",
    "    html = ''\n",
    "    for video_path, text in zip(video_path_list, text_list):\n",
    "        html_line = '<img src=\"{}\"> {} <br/><br/>'.format(video_path, text)\n",
    "        html += html_line\n",
    "    return display.HTML(html)\n",
    "\n",
    "def convert_video2gif(video_path, output_gif_path, start_time=0.0, end_time=1000.0, num_samples=16):\n",
    "    p = (\n",
    "        pipe.input('video_file')\n",
    "        .flat_map('video_file', 'frame', ops.video_decode.ffmpeg(start_time=start_time, end_time=end_time, sample_type='time_step_sample', args={'time_step': 3}))\n",
    "             .output('frame')\n",
    "    )\n",
    "    frames = p(video_path).to_list()\n",
    "    imgs = [PILImage.fromarray(frame[0]) for frame in frames]\n",
    "    imgs = [img.resize((int(img.width/6), int(img.height/6)), PILImage.NEAREST) for img in imgs]\n",
    "    imgs[0].save(fp=output_gif_path, format='GIF', append_images=imgs[1:], save_all=True, loop=0)\n",
    "\n",
    "\n",
    "def display_gifs_from_video(video_path_list, text_list, start_time_list = None, end_time_list = None, tmpdirname = './tmp_gifs'):\n",
    "    Path(tmpdirname).mkdir(exist_ok=True)\n",
    "    gif_path_list = []\n",
    "    for i, video_path in enumerate(video_path_list):\n",
    "        video_name = str(Path(video_path).name).split('.')[0]\n",
    "        gif_path = Path(tmpdirname) / (video_name + '.gif')\n",
    "        if start_time_list is not None:\n",
    "            convert_video2gif(video_path, gif_path, start_time=start_time_list[i], end_time=end_time_list[i])\n",
    "        else:\n",
    "            convert_video2gif(video_path, gif_path)\n",
    "        gif_path_list.append(gif_path)\n",
    "    return display_gif(gif_path_list, text_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "279391c4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"tmp_gifs/d62ce5becff14a0c9c7dab5eea6647dc_the_wandering_earth_Wu_Jing_became_teenage_idol-1qf4y1G7gM.gif\"> ./VCSL-demo-en/the_wandering_earth/d62ce5becff14a0c9c7dab5eea6647dc_the_wandering_earth_Wu_Jing_became_teenage_idol-1qf4y1G7gM.flv <br/><br/><img src=\"tmp_gifs/e5dc80abd7a24b47accde190c9fdbcdc-The_Wandering_Earth_is_on_CCTV-1db411m7f8.gif\"> ./VCSL-demo-en/the_wandering_earth/e5dc80abd7a24b47accde190c9fdbcdc-The_Wandering_Earth_is_on_CCTV-1db411m7f8.flv <br/><br/><img src=\"tmp_gifs/ef65e0f662e646a88a13b6eddb640e48-News_Broadcast_on_Wandering_Earth-1xb411U7uE.gif\"> ./VCSL-demo-en/the_wandering_earth/ef65e0f662e646a88a13b6eddb640e48-News_Broadcast_on_Wandering_Earth-1xb411U7uE.flv <br/><br/>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import random\n",
    "random.seed(9)\n",
    "vcsl_demo_root = './VCSL-demo-en/'\n",
    "\n",
    "event_list = os.listdir(vcsl_demo_root)\n",
    "\n",
    "random_event = random.choice(event_list)\n",
    "random_event_folder = os.path.join(vcsl_demo_root, random_event)\n",
    "random_event_videos = [os.path.join(random_event_folder, video_file) for video_file in os.listdir(random_event_folder)]\n",
    "tmpdirname = './tmp_gifs'\n",
    "display_gifs_from_video(random_event_videos, random_event_videos, tmpdirname=tmpdirname)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "17d27c31",
   "metadata": {},
   "outputs": [],
   "source": [
    "import towhee\n",
    "\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = '1'\n",
    "\n",
    "def merge_ndarray(x):\n",
    "    import numpy as np\n",
    "    return np.concatenate(x).reshape(-1, x[0].shape[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a001b449",
   "metadata": {},
   "source": [
    "### Setup Milvus and create a Milvus Collection\n",
    "\n",
    "The last thing to be prepared is Milvus. For more options & detailed instructions, you can refer to [Milvus doc](https://milvus.io). If you need more help for Milvus, feel free to submit tickets or join discussion in [Milvus github](https://github.com/milvus-io/milvus).\n",
    "\n",
    "Please make sure that you have started a [Milvus service](https://milvus.io/docs/install_standalone-docker.md). This notebook uses [milvus 2.2.10](https://milvus.io/docs/v2.2.x/install_standalone-docker.md) and [pymilvus 2.2.11](https://milvus.io/docs/release_notes.md#2210)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "c1926c1d",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Download docker yaml for Milvus standalone\n",
    "! wget https://github.com/milvus-io/milvus/releases/download/v2.2.10/milvus-standalone-docker-compose.yml -O docker-compose.yml\n",
    "# Run command below under the same directory as the docker yaml\n",
    "! docker-compose up -d\n",
    "# Install pymilvus\n",
    "! python -m pip install pymilvus==2.2.11"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62c070a4",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Let's first create a `video_deduplication` collection that uses the [L2 distance metric](https://milvus.io/docs/metric.md#Euclidean-distance-L2) and an [IVF_FLAT index](https://milvus.io/docs/index.md#IVF_FLAT)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "70ccbb06",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pymilvus import connections, FieldSchema, CollectionSchema, DataType, Collection, utility\n",
    "\n",
    "connections.connect(host='127.0.0.1', port='19530')\n",
    "\n",
    "\n",
    "def create_milvus_collection(collection_name, dim):\n",
    "    if utility.has_collection(collection_name):\n",
    "        utility.drop_collection(collection_name)\n",
    "\n",
    "    fields = [\n",
    "        FieldSchema(name='id', dtype=DataType.INT64, descrition='the id of the embedding', is_primary=True, auto_id=True),\n",
    "        FieldSchema(name='path', dtype=DataType.VARCHAR, descrition='the path of the embedding', max_length=500),\n",
    "        FieldSchema(name='embedding', dtype=DataType.FLOAT_VECTOR, descrition='video embedding vectors', dim=dim)\n",
    "    ]\n",
    "    schema = CollectionSchema(fields=fields, description='video dedup')\n",
    "    collection = Collection(name=collection_name, schema=schema)\n",
    "\n",
    "    index_params = {'metric_type': 'IP', 'index_type': \"IVF_FLAT\", 'params': {\"nlist\": 1}}\n",
    "    collection.create_index(field_name=\"embedding\", index_params=index_params)\n",
    "    return collection\n",
    "\n",
    "collection = create_milvus_collection('video_copy_detection', 256)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09dfe584",
   "metadata": {},
   "source": [
    "## Video Copy Detection\n",
    "\n",
    "In this section, we'll show how to build our Video Copy Detection engine using Milvus. The basic idea behind Video Copy Detection is the extract embeddings from videos using Deep Neural Network and store them in Milvus, then get query videos embeddings and compare with those stored in Milvus.\n",
    "\n",
    "We use [Towhee](https://towhee.io/), a machine learning framework that allows for creating data processing pipelines. [Towhee](https://towhee.io/) also provides predefined operators which implement insert and query operation in Milvus.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09f42898",
   "metadata": {},
   "source": [
    "### Load Video Embeddings into Milvus\n",
    "\n",
    "For every video, we decode it to image frames, and then using neural network to extract their embeddings. We insert them to Milvus and levelDB for storage.\n",
    "![](video_decopy_insert.png)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ed5d8e4e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 7min 1s, sys: 9.84 s, total: 7min 11s\n",
      "Wall time: 3min 12s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "from towhee import pipe, ops\n",
    "import glob\n",
    "\n",
    "emb_pipe = (\n",
    "    pipe.input('url')\n",
    "        .flat_map('url', 'frames', ops.video_decode.ffmpeg(sample_type='time_step_sample', args={'time_step': 1}))\n",
    "        .map('frames', 'emb', ops.image_embedding.isc())\n",
    "        .map(('url', 'emb'), 'insert_res', ops.ann_insert.milvus_client(host='127.0.0.1', port='19530', collection_name='video_copy_detection'))\n",
    "        .window_all('emb', 'video_emb', merge_ndarray)\n",
    "        .map(('url', 'video_emb'), ('url_vec_status'), ops.kvstorage.insert_hbase('127.0.0.1', 9090, 'video_copy_detection'))\n",
    "        # .map(('url', 'video_emb'), ('url_vec_status'), ops.kvstorage.insert_leveldb('url_vec.db'))\n",
    "        .output()\n",
    ")\n",
    "\n",
    "path = glob.glob('VCSL-demo-en/*/*')\n",
    "for i in path:\n",
    "    result = emb_pipe(i)\n",
    "\n",
    "del emb_pipe"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35fa1c5b",
   "metadata": {},
   "source": [
    "### Query videos\n",
    "\n",
    "In theory, for each query video, it is necessary to match and retrieve all the videos in the database, which will cause huge overhead. In this tutorial, we perform a rough video selection which filter the videos with low similarity to solve this problem. \n",
    " \n",
    "First, for every query frame, we retrieve a certain number of similar frames through Milvus, which match for a specified video. The videos of these frames are then aggregated, sorted, and filtered. Then, the video embeddings of remaining videos and embedding of the query video are processed for localizing copyed segments. In this way, we can filter out videos with low similarity, saving a lot of computation for the whole pipeline.\n",
    "![](video_decopy_query.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d85fbb0b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">url</th> <th style=\"text-align: center; font-size: 130%; border: none;\">candidates</th> <th style=\"text-align: center; font-size: 130%; border: none;\">similar_segment</th> <th style=\"text-align: center; font-size: 130%; border: none;\">segment_score</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[0, 0, 52, 52]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[1.0192308185192256] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[0, 1, 19, 25]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.6052339520565299] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[0, 143, 22, 164]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.7498454105022342] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/baisui_mountain/03584e404c0847fcbe5f9c486e8f8fc7-Baisui_Mountain_means_this-1UE411V74D.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/baisui_mountain/217a12c936414660a53b55b22e2aea59-20200607Baisui_Mountain-1kf4y127GB.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">url</th> <th style=\"text-align: center; font-size: 130%; border: none;\">candidates</th> <th style=\"text-align: center; font-size: 130%; border: none;\">similar_segment</th> <th style=\"text-align: center; font-size: 130%; border: none;\">segment_score</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[0, 0, 167, 167]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[1.0059880678525228] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[136, 0, 164, 25],[138, 43, 166, 79]] len=2</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.45078764096745905,0.2279533192049712] len=2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[141, 0, 164, 21]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.695049833167683] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/baisui_mountain/03584e404c0847fcbe5f9c486e8f8fc7-Baisui_Mountain_means_this-1UE411V74D.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/the_wandering_earth/ef65e0f662e646a88a13b6eddb640e48-News_Broadcast_on_Wandering_Earth-1xb411U7uE.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">url</th> <th style=\"text-align: center; font-size: 130%; border: none;\">candidates</th> <th style=\"text-align: center; font-size: 130%; border: none;\">similar_segment</th> <th style=\"text-align: center; font-size: 130%; border: none;\">segment_score</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[0, 0, 83, 83]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[1.012048258120755] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[1, 1, 25, 19],[23, 0, 76, 23]] len=2</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.6449160008203416,0.3358775589026903] len=2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[1, 144, 25, 164],[40, 150, 83, 166]] len=2</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.6091001223434102,0.3373360962180768] len=2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/yellow_alien/0fba4debd48245699971c4c18608cde3-Yellow_skinned_alien_low_cost_production-1vP4y1s79h.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 3min 5s, sys: 1min 14s, total: 4min 20s\n",
      "Wall time: 49.8 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "from towhee.datacollection import DataCollection\n",
    "collection.load()\n",
    "\n",
    "search_pipe = (\n",
    "    pipe.input('url')\n",
    "        .flat_map('url', 'frames', ops.video_decode.ffmpeg(sample_type='time_step_sample', args={'time_step': 1}))\n",
    "        .map('frames', 'emb', ops.image_embedding.isc())\n",
    "        .flat_map('emb', 'res', ops.ann_search.milvus_client(host='127.0.0.1', port='19530', collection_name='video_copy_detection', limit=5, output_fields=['path'], metric_type='IP'))\n",
    "        .window_all('res', ('retrieved_urls', 'score'), lambda x: ([i[2] for i in x], [i[1] for i in x]))\n",
    "        .window_all('emb', 'video_emb', merge_ndarray)\n",
    "        .flat_map(('retrieved_urls','score'), 'candidates', ops.video_copy_detection.select_video(top_k=5, reduce_function='sum', reverse=True))\n",
    "        # .map('candidates', 'retrieved_emb', ops.kvstorage.from_leveldb(path = 'url_vec.db', is_ndarray=True))\n",
    "        .map('candidates', 'retrieved_emb', ops.kvstorage.search_hbase('127.0.0.1', 9090, 'video_copy_detection', is_ndarray=True))\n",
    "        .map(('video_emb', 'retrieved_emb'), ('similar_segment', 'segment_score'), ops.video_copy_detection.temporal_network(min_length=1))\n",
    "        .output('url', 'candidates', 'similar_segment', 'segment_score')\n",
    ")  \n",
    "\n",
    "path = glob.glob('VCSL-demo-en/madongmei/*')\n",
    "for i in path:\n",
    "    result = search_pipe(i)\n",
    "    DataCollection(result).show()\n",
    "\n",
    "del search_pipe"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "983fa56d",
   "metadata": {},
   "source": [
    "For each frame of each query video, we query the most similar frame information of 5 frames from Milvus. We aggregate and sort this information, and select candidate videos with topk=5. For this query video and the corresponding 5 candidate videos, `temporal_network` calculation is performed, and finally the detected duplicate segments are obtained. \n",
    "\n",
    "Note that our query uses the same dataset, in which there are 5 events, and each event has 3 videos, which are copies of each other. Using this dataset to query itself means that for each video, the correct query result should be the three videos under its own event. \n",
    "\n",
    "The output `similar_segment` column is detected segments list, which format is list of `[query_start_second, ref_start_second, query_end_second, ref_end_second]`. And `segment_score` column is the corresponding similarity score of each segment. We can observe that each query video does detect only 3 results of its own event, which is consistent with ground truth.\n",
    " \n",
    "Let's take the result of the following line as an example, with `similar_segment` = [0, 143, 22, 164], indicating that in the query video, from 0 to 22 seconds, and the ref video from 143 to 164 seconds are repeated. We can display these clips.\n",
    "![example](example.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8d2cb13e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/f1/bqcl4tcj7z17llp4l9215lph0000gn/T/ipykernel_8757/2347910899.py:21: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.\n",
      "  imgs = [img.resize((int(img.width/6), int(img.height/6)), PILImage.NEAREST) for img in imgs]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<img src=\"tmp_gifs2/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.gif\"> VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv <br/><br/><img src=\"tmp_gifs2/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.gif\"> VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv <br/><br/>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "event_videos = ['VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv',\n",
    "               'VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv']\n",
    "tmpdirname2 = './tmp_gifs2'\n",
    "\n",
    "display_gifs_from_video(event_videos, event_videos, start_time_list=[0, 143], end_time_list=[22, 164], tmpdirname=tmpdirname2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b410548",
   "metadata": {},
   "source": [
    "### Options\n",
    "\n",
    "For image embedding operator and kv storage operator, some options are provided:\n",
    "\n",
    "- Image embedding operator: Check [Towhee Image Embedding](https://towhee.io/tasks/detail/operator?field_name=Computer-Vision&task_name=Image-Embedding) for more pre-trained models Towhee encapsulates as operators. For video copy detection task, we recommend `ISC`. `ISC` is a pre-trained model that works pretty well for such task. Besides, we support following models from timm:\n",
    "\n",
    "\t'isc',\n",
    "\t'gmixer_24_224',\n",
    "\t'resmlp_12_224',\n",
    "\t'coat_lite_mini',\n",
    "\t'deit_small_patch16_224',\n",
    "\t'pit_xs_224',\n",
    "\t'convit_small',\n",
    "\t'tnt_s_patch16_224',\n",
    "\t'pit_ti_224',\n",
    "\t'resmlp_36_distilled_224',\n",
    "\t'convit_tiny',\n",
    "\t'coat_lite_small',\n",
    "\t'coat_lite_tiny',\n",
    "\t'deit_tiny_patch16_224',\n",
    "\t'cait_xxs24_224',\n",
    "\t'cait_s24_224',\n",
    "\t'cait_xxs36_224',\n",
    "\t'vit_small_patch32_224',\n",
    "\t'vit_small_patch32_384',\n",
    "\t'vit_small_r26_s32_224',\n",
    "\t'vit_small_patch16_224'.\n",
    "\n",
    "\tNote that the vector dimension of the milvus collection we create before should change accordingly depends on the output embedding shape of the embedding model (e.g. 384 for `gmixer_24_224`, 256 for `ISC`);\n",
    "\n",
    "\n",
    "- kv storage: Check [Towhee kv storage](https://towhee.io/kvstorage) for different kv database. If one wants to run pipeline on some large dataset, we recommend `hbase`, otherwise `leveldb` would be enough. If we choose leveldb as kv database, we need to del the pipeline after running to release the leveldb lock."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21c345db",
   "metadata": {},
   "source": [
    "### Towhee Built-in Pipeline\n",
    "\n",
    "For users's convenience, Towhee has excapsulate several built-in pipelines including `video_embedding` and `video_copy_detection`. So one can create and run the pipeline above within a few lines of code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f3361e0c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from towhee import AutoPipes, AutoConfig\n",
    "\n",
    "emb_conf = AutoConfig.load_config('video_embedding')\n",
    "emb_conf.collection = 'video_copy_detection'\n",
    "# emb_conf.leveldb_path = 'url_vec.db'\n",
    "emb_conf.hbase_table = 'video_copy_detection'\n",
    "emb_conf.device = 0\n",
    "emb_pipe = AutoPipes.pipeline('video_embedding', emb_conf)\n",
    "\n",
    "path = glob.glob('VCSL-demo-en/*/*')\n",
    "\n",
    "for i in path:\n",
    "    result = emb_pipe(i)\n",
    "\n",
    "del emb_pipe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "defb8c1c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">url</th> <th style=\"text-align: center; font-size: 130%; border: none;\">candidates</th> <th style=\"text-align: center; font-size: 130%; border: none;\">similar_segment</th> <th style=\"text-align: center; font-size: 130%; border: none;\">segment_score</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[0, 1, 19, 25]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.6052339409672937] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[0, 0, 52, 52]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[1.0192307784007146] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[0, 143, 22, 164]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.7498453827791436] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/animation/043bee7a71f347f18e8576bb1a01c86b-Langrisser-y3qAPvWnL18.mkv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/the_wandering_earth/d62ce5becff14a0c9c7dab5eea6647dc_the_wandering_earth_Wu_Jing_became_teenage_idol-1qf4y1G7gM.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">url</th> <th style=\"text-align: center; font-size: 130%; border: none;\">candidates</th> <th style=\"text-align: center; font-size: 130%; border: none;\">similar_segment</th> <th style=\"text-align: center; font-size: 130%; border: none;\">segment_score</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[0, 0, 167, 167]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[1.0059880193122133] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[136, 0, 164, 25],[138, 43, 166, 79]] len=2</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.45078762972129965,0.22795329638756812] len=2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[141, 0, 164, 21]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.6950498060746626] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/animation/043bee7a71f347f18e8576bb1a01c86b-Langrisser-y3qAPvWnL18.mkv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/the_wandering_earth/ef65e0f662e646a88a13b6eddb640e48-News_Broadcast_on_Wandering_Earth-1xb411U7uE.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">url</th> <th style=\"text-align: center; font-size: 130%; border: none;\">candidates</th> <th style=\"text-align: center; font-size: 130%; border: none;\">similar_segment</th> <th style=\"text-align: center; font-size: 130%; border: none;\">segment_score</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[0, 0, 83, 83]] len=1</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[1.0120482329862663] len=1</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/0640bd5d43d1499c962e275be6b804ef-Does_MaDongmei_live_here-1e64y1y799.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[1, 144, 25, 164],[40, 150, 83, 166]] len=2</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.609100111506202,0.3373360800541053] len=2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/ad244c924f31461a9d809c77ae251ac1-the_classic_dialogue_what_is_Ma_Mei-1y7411n7y1.flv</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[[1, 1, 25, 19],[23, 0, 76, 23]] len=2</td> <td style=\"text-align: left; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">[0.6449159866287595,0.3358775534127888] len=2</td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/animation/043bee7a71f347f18e8576bb1a01c86b-Langrisser-y3qAPvWnL18.mkv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/madongmei/8ad81fc9fe0a47dbaab1b4cdc40bf07b-go_and_cool_off_divine_comedy_Ma_Dongmei-1t54y117JK.flv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">VCSL-demo-en/animation/1b4e048011714eab928650f64e6370a6-123-JqSBkvPsYBw.mkv</td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td> <td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \"></td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from towhee.datacollection import DataCollection\n",
    "\n",
    "connections.connect(host='127.0.0.1', port='19530')\n",
    "Collection('video_copy_detection').load()\n",
    "\n",
    "search_conf = AutoConfig.load_config('video_copy_detection')\n",
    "search_conf.collection = 'video_copy_detection'\n",
    "# search_conf.leveldb_path = 'url_vec.db'\n",
    "search_conf.hbase_table = 'video_copy_detection'\n",
    "search_conf.topk = 5\n",
    "search_conf.device = 0\n",
    "search_pipe = AutoPipes.pipeline('video_copy_detection', search_conf)\n",
    "\n",
    "path = glob.glob('VCSL-demo-en/madongmei/*')\n",
    "\n",
    "for i in path:\n",
    "    result = search_pipe(i)\n",
    "    DataCollection(result).show()\n",
    "\n",
    "del search_pipe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "10fadca9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import shutil, os\n",
    "from pathlib import Path\n",
    "\n",
    "shutil.rmtree('VCSL-demo-en')\n",
    "if Path('url_vec.db').exists():\n",
    "\tshutil.rmtree('url_vec.db')\n",
    "os.remove('VCSL-demo-en.zip')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8bb659ec",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "f7dd10cdbe9a9c71f7e71741efd428241b5f9fa0fecdd29ae07a5706cd5ff8a2"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}